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Abstract  In  this  paper  we  look  at  the  phenomenon  that  is 
the  Turing  test.  We  consider  how  Turing  originally  intro¬ 
duced  his  imitation  game  and  discuss  what  this  means  in  a 
practical  scenario.  Due  to  its  popular  appeal  we  also  look 
into  different  representations  of  the  test  as  indicated  by 
numerous  reviewers.  The  main  emphasis  here,  however,  is 
to  consider  what  it  actually  means  for  a  machine  to  pass  the 
Turing  test  and  what  importance  this  has,  if  any.  In  par¬ 
ticular  does  it  mean  that,  as  Turing  put  it,  a  machine  can 
“think”.  Specifically  we  consider  claims  that  passing  the 
Turing  test  means  that  machines  will  have  achieved 
human-like  intelligence  and  as  a  consequence  the  singu¬ 
larity  will  be  upon  us  in  the  blink  of  an  eye. 

Keywords  Deception  detection  •  Natural  language  • 
Turing’s  imitation  game  •  Chatbots  •  Machine 
misidentification 

Introduction 

There  are  those  who  believe  that  passing  the  Turing  test 
means  that  human-level  intelligence  will  have  been 
achieved  by  machines  [10].  The  direct  consequence  of  this, 
as  pointed  out  by  Kurzweil  [11]  and  others,  is  that  the 
singularity  will  be  upon  us,  thereby  resulting  in  the  demise 
of  the  human  race.  In  this  paper  we  do  not  wish  to  dispute 
the  latter  of  these  arguments,  dramatic  though  it  is.  What 
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we  do  wish  to  dispel,  however,  is  the  assumption  which 
links  passing  the  Turing  test  with  the  achievement  for 
machines  of  human-like  or  human-level  intelligence. 

Unfortunately  the  assumed  chain  of  events  which  means 
that  passing  the  Turing  test  sounds  the  death  knell  for 
humanity  appears  to  have  become  engrained  in  the  thinking 
in  certain  quarters.  One  interesting  corollary  of  this  is  that 
when  it  was  announced  in  2014  that  the  Turing  test  had 
been  finally  passed  [39]  there  was  an  understandable 
response  from  those  same  quarters  that  it  was  not  possible 
for  such  an  event  to  have  occurred,  presumably  because  we 
were  still  here  in  sterling  health  to  both  make  and  debate 
the  pronouncement.  Interestingly  the  main  academic 
argument  which  was  thrown  up  was  that  the  machine 
which  passed  the  test  did  not  exhibit  human-like  intelli¬ 
gence,  and  therefore,  the  test  could  not  have  been  passed. 
Consider  this,  for  example,  from  Murray  Shanahan  of 
Imperial  College  London:  “Of  course  the  Turing  Test 
hasn’t  been  passed... We  are  still  a  very  long  way  from 
achieving  human-level  AI”  [10]. 

It  is  therefore,  we  feel,  of  vital  importance  that  we 
look  at  various  aspects  of  this  question.  Because  if  Murray 
Shanahan  and  Ray  Kurzweil  and  their  colleagues  are  cor¬ 
rect  then  the  developers  of  the  computer  programmes 
which  compete  in  the  Turing  test  are,  if  they  are  successful, 
about  to  put  an  end  to  the  human  race.  So  shouldn’t  we  do 
something  about  such  developers,  maybe  lock  them  up, 
well  away  from  any  laptop  in  case  they  design  a  pro¬ 
gramme  of  destruction.  On  the  other  hand  dare  we  suggest 
that  either  Shanahan  or  Kurzweil  is  incorrect? 

The  singularity  [11]  is  an  event  dependent  on  the  overall 
improvement  and  power  of  Artificial  Intelligence  where 
intelligent  machines  can  design  successive  generations  of 
increasingly  more  powerful  machines,  eventually  creating 
intelligence  that  firstly  is  equivalent  to  that  of  humans  and 
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then  surpasses  it.  Indeed  the  capabilities  of  such  an  Arti¬ 
ficial  Intelligence  may  well  be  impossible  for  a  human  to 
comprehend.  The  singularity  is  the  point  beyond  which 
events  are  beyond  the  control  of  humans,  resulting  either  in 
humans  upgrading  (with  implants)  to  become  Cyborgs  or 
with  intelligent  machines  taking  control.  Either  way,  it’s 
not  good  news  for  ordinary  humans. 

Taking  a  sensible  look  at  this  issue,  and  someone  needs 
to,  we  wish  to  analyse  why,  with  the  “standard  Turing  test” 
(Fig.  lb)  (defined  as  5  min,  unrestricted  question-answer 
simultaneous  comparison  version  [18,  25] — having  been 
passed — more  than  30  %  of  the  human  interrogators  fail  to 
correctly  identify  the  machine)  we  are  all  still  here  and  this 
paper  can  be  read  by  (presumably)  humans.  The  flaw  in  the 
Shanahan/Kurzweil  argument,  at  this  time,  we  contest  is 
that  Shanahan  is  just  plain  wrong.  Passing  the  Turing  test 
has  no  relationship  with  human-like  intelligence  (or  AI) 
other  than  in  the  sense  of  a  machine  possibly  being  rea¬ 
sonably  effective  in  its  own  version  of  human  conversation 
for  a  sustained  short  period,  over  which  time  it  has  proved 
to  be  successful  in  fooling  a  collection  of  humans.  Kurz- 
weiTs  singularity  argument  may  or  may  not  also  be  wrong, 
but  that’s  not  what  we  wish  to  discuss  here.  The  point  is 
that  as  long  as  one  of  the  Shanahan/Kurzweil  pair  is  wrong 
then  the  human  race  is  still  looking  good  (apart  from  its 
multitude  of  other  problems  that  is). 

What  we  wish  to  do  in  this  paper  is  to  take  a  look  at 
what  the  Turing  test  actually  is,  as  stipulated/set  out  by 
Alan  Turing,  rather  than  to  consider  some  related  test 
which  some  might  wish  to  call  the  Turing  test  or  what 
someone  might  want  the  test  to  be,  because  they’ve  thought 
of  a  different/better  test.  We  acknowledge  here  that  dif¬ 
ferent/better  tests  of  computer  ability,  even  in  terms  of  only 
conversation,  exist  but  again  they  are  not  the  subject  of  this 


paper.  So  we  stick  as  closely  as  possible  to  what  the  test  is, 
based  entirely  on  Turing’s  own  words.  We  acknowledge, 
however,  that  there  are  different  interpretations  of  the  test, 
whether  each  test  should  last  for  5  or  10  min  for  example 
or  even  if  Turing  intended  the  test  as  some  sort  of  mind 
modelling  exercise.  However,  none  of  these,  we  argue, 
result  in  the  end  of  humanity.  Indeed  Turing  himself  said 
that  humans  would  be  needed  to  maintain  the  machines 
[28]. 

We  then  subsequently  present  some  example  discourses, 
taken  from  a  series  of  tests  held  at  the  Royal  Society  in 
2014.  One  of  these  involves  the  machine  Eugene  Goostman 
which  actually  passed  the  test  at  that  event.  Following  this 
we  look  at  some  ways  in  which  machines  can  pass  the  test, 
as  it  has  been  defined  in  terms  of  the  standard  definition 
[26].  Finally  we  draw  some  conclusions.  One  of  which,  and 
some  might  argue  perhaps  the  most  important,  is  that 
humanity  is  not  about  to  expire. 

To  be  clear  though  we  are  aware  that  different  theories 
regarding  the  Turing  test  and  its  meaning  exist  and  that 
other  theories  have  been  put  forward  along  the  lines  that 
machines  will  not  take  over  from  humans.  In  this  paper  we 
are  explicitly  only  concerned  with  the  pairing  of  statements 
that  says  (a)  passing  the  Turing  test  means  that  human- 
level  intelligence  will  have  been  achieved  in  AI  and 
(b)  when  AI  exhibits  human-level  intelligence  that  will 
mean  the  end  of  humanity  as  we  know  it.  We  are  only  too 
aware,  for  example,  that  in  describing  his  test,  Turing 
discussed  men  and  women  as  hidden  entities  and  the  pos¬ 
sibility  of  gender  blur.  Whilst  this  is  extremely  interesting, 
it  is  not  what  we  wish  to  look  at  in  this  paper.  We  focus 
here  entirely  on  one  specific  issue  which  is  that  if  both 
Shanahan  and  Kurzweil  are  correct  then  a  machine  passing 
the  Turing  test  means  that  humanity  is  doomed! 


Fig.  1  Turing’s  two  tests  for  his  imitation  game:  Left  a  one-to-one;  Right  b  one  judge-two  hidden  interlocutors 
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The  Turing  Test 

In  his  1950  paper  entitled  “Computing  Machinery  and 
Intelligence”  [30],  Alan  Turing  started  by  considering  the 
question,  “Can  machines  think?”  However,  rather  than  get 
bogged  down  with  definitions  of  both  of  the  words  “ma¬ 
chine”  and  “think”  he  replaced  the  question  with  one 
based  on  a  much  more  practical  scenario,  namely  his 
imitation  game.  The  game  has  since  become  more  widely 
known,  particularly  in  the  popular  domain,  as  the  Turing 
test.  He  did  not,  however,  at  any  point,  refer  to  his  test/ 
game  as  being  any  indication  of  intelligence,  human-like  or 
otherwise. 

Turing  [31]  described  the  game  as  follows:  “The  idea  of 
the  test  is  that  a  machine  has  to  try  and  pretend  to  be  a  man, 
by  answering  questions  put  to  it,  and  it  will  only  pass  if  the 
pretence  is  reasonably  convincing.  A  considerable  portion 
of  a  jury,  who  should  not  be  expert  about  machines,  must 
be  taken  in  by  the  pretence”  [4].  So  Turing  spoke  here  of  a 
jury  (nominally  12)  as  opposed  to  the  “average  interroga¬ 
tors”  he  mentioned  in  his  1950  paper  [30],  as  we  will  see 
shortly.  Importantly  he  also  spoke  of  a  machine  “passing” 
the  test  and  that  the  interrogators  should  not  be  experts. 
Interestingly,  however,  we  do  include  a  transcript  later  in 
which  a  machine  did  fool  an  expert  into  thinking  that  it  was 
human. 

Turing’s  imitation  game  is  described  as  an  experiment 
that  can  be  practicalised  in  two  different  ways  (see  Fig.  1) 
[17]: 

1.  one  interrogator-one  hidden  interlocutor  (Fig.  la), 

2.  one  interrogator-two  hidden  interlocutors  (Fig.  lb). 

In  both  cases  the  machine  must  provide  “satisfactory” 
and  “sustained”  answers  to  any  questions  put  to  it  by  the 
human  interrogator  [30,  p.  447]. 

Of  the  types  of  test  looked  at  here,  the  3-participant  tests 
have  previously  been  shown  to  be  stricter  tests,  i.e.  more 
difficult  for  machines,  than  2-participant  tests  in  which  an 
interrogator  converses  with  only  one  hidden  entity,  either  a 
human  or  machine,  at  a  time  [22] .  For  the  main  arguments 
set  out  in  this  paper,  the  results  apply  to  either  type  of  test. 

Turing  did  not  explicitly  state  specific  rules  for  his  test 
in  a  paragraph  headed  “Rules  for  my  test”  or  some  such 
like,  and  hence  what  is  required  of  a  machine  in  order  to 
pass.  What  he  did  clearly  state  in  his  1950  paper,  and  which 
we  contest  amounts  to  the  same  thing,  was  as  follows:  “I 
believe  that  in  about  50  years’  time  it  will  be  possible,  to 
programme  computers  to  make  them  play  the  imitation 
game  so  well  that  an  average  interrogator  will  not  have 
more  than  70  %  chance  of  making  the  right  identification 
after  5  min  of  questioning”  [30].  Having  clearly  spelt  out 


the  imitation  game,  this  would  appear  to  be  direction 
enough  from  Turing. 

Although  this  appeared  to  have  been  written  more  in  the 
sense  of  a  prediction,  it  is  the  only  place  where  Turing 
directly  stated  parameters  for  his  game/test,  with  a  clear 
hurdle  to  be  met  in  terms  of  performance.  To  put  this  more 
simply,  for  a  machine  to  pass  the  Turing  test,  in  all  of  the 
tests  in  which  a  machine  takes  part,  the  interrogators  must 
make  the  wrong  identification  (i.e.  not  the  right  identifi¬ 
cation)  30  %  or  more  of  the  time  after,  in  each  case,  5-min- 
long  conversations.  We  can  take  it  directly  that  the  wrong 
identification  is  anything  other  than  the  right  identification. 
Also,  because  Turing  spoke  of  a  Jury  we  can  understand 
from  that  that  at  least  twelve  judges/interrogators  must  be 
able  to  test  a  machine  in  their  own  way/style.  But  also  that 
hundreds  of  judges  are  not  a  requirement,  a  jury  is  appro¬ 
priate  and  will  suffice. 

We  will  shortly  look  at  what  is  meant  by  the  “right 
identification”,  as  this  is  critical.  However,  we  can  take  it 
immediately  that  Turing  set  the  challenge  as  a  5 -min 
exercise,  no  more  and  no  less.  At  no  other  point  in  Turing’s 
papers  did  he  mention  any  other  time  duration  for  his  tests. 
In  general  we  can  experience  that  the  longer  tests  last  so 
the  more  difficult  it  is  for  a  machine  to  satisfactorily  pre¬ 
tend  to  be  a  human.  Indeed  given  the  technology  we  have 
at  present,  5  min  would  appear  to  be  an  appropriate  chal¬ 
lenge.  In  a  20-min  test,  at  this  time  in  computer  natural 
language  development,  it  is  extremely  difficult  for  a 
machine  to  fool  a  human  interrogator  over  that  period  into 
thinking  that  it’s  a  human. 

It  is  widely  recognised  that  getting  machines  to  achieve, 
or  at  least  appear  to  achieve,  human-like  responses  is  a 
difficult  task  [5,  32].  Even  in  terms  of  the  Turing  test,  based 
purely  on  conversation,  taking  into  account  such  issues  as 
what  knowledge  is  brought  to  the  table  and  assumed  [34]  or 
whether  one  of  the  entities  is  lying  [35]  can  completely 
change  an  appearance.  There  are  also  numerous  strategies 
that  can  be  employed  by  machines  in  order  to  successfully 
fool  an  interrogator  [36]. 

One  fuzzy  issue,  however,  is  did  Turing  mean  5  min  in 
total  for  a  parallel  paired  3 -participant  conversation  or  rather 
allowing  an  average  of  5  min  each,  hence  a  total  of  10  min, 
for  the  two  hidden  entities  involved  [23]?  Michie  [14] 
interpreted  the  test  as  approximately  2  Vi-min  interrogation 
per  entity  in  a  pair.  However,  in  practice  the  conversation  is 
rarely  balanced  exactly.  For  all  of  the  practical  tests  which 
we  have  organised,  a  time  limit  of  5  min,  as  stated  by  Turing 
himself,  has  been  placed,  because  the  current  state  of  con¬ 
versational  technology  is  not  ready  for  longer  duration  tests. 
That  said,  we  acknowledge  the  potential  validity  of  the 
alternative,  which  we  will  call  here  the  Sloman  view. 
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Whether  it  is  Michie,  Sloman  or  ourselves  who  reads 
this  one  correctly  is  a  relatively  insignificant  point  in  the 
big  argument.  Otherwise  we  would  be  in  the  laughable 
state  which  says  OK  a  machine  can  fool  you  into  thinking 
they  are  human  over  a  5-min  conversation  but  they  can’t  do 
so  over  10  min  therefore  we’re  all  saved  and  humanity  can 
go  on.  Scientifically  this  would  mean  there  must  be  a 
conversation  time  somewhere  between  5  and  10  min  such 
that  once  it  is  achieved  by  a  machine,  we’re  all  doomed. 

It  is  also  interesting  that  in  the  2-participant  test  an 
interrogator  spends  all  5  min  conversing  with  one  machine 
only  whereas  in  the  3 -participant  test  the  average  time 
spent  with  each  hidden  entity  is  clearly  2.5  min.  Despite 
this  the  3-participant  test,  the  one  Turing  spoke  of  in  1950 
[30],  is  the  more  difficult  for  machines  to  achieve  good 
results,  most  likely  because  of  the  direct,  parallel  com¬ 
parison  that  occurs  in  such  cases. 

It  is  worth  remembering  though  that  in  either  type  of  test 
an  interrogator,  in  an  actual  “official”  Turing  test,  when 
communicating  with  a  machine,  does  not  know  at  that  time 
that  it  is  in  fact  a  machine,  indeed  it  is  a  decision  about  its 
nature  that  they  have  to  come  to.  This  is  a  critical  point  and 
is  one  of  the  main  features  of  the  test.  Such  a  situation  is,  as 
you  might  guess,  far  different  to  the  case  when  an  inter¬ 
rogator  knows  for  certain  that  they  are  communicating  with 
a  machine,  as  in  the  case  of  an  online  hot  [1].  Despite  this 
vital  point,  for  some  reason  there  are  a  number  of  people 
who  completely  ignore  this  critical  aspect  of  the  test,  go 
online  to  converse  with  a  hot,  which  they  already  know  to 
be  a  hot,  and  declare  in  conclusion  that  it  is  obviously  a  hot 
[16].  Clearly  some  education  is  required  as  to  what  the 
Turing  test  actually  involves. 

However,  this  is  somewhat  akin  to  the  Oxford  Univer¬ 
sity  Philosophy  Professor  and  his  students  who  took  part  in 
9  actual  Turing  tests  in  2008  and  then  went  to  academic 
print  in  claiming  it  was  easy  to  spot  which  were  the 
machines  and  which  were  the  humans  in  all  the  tests  in 
which  they  were  involved;  indeed  they  published  this  in  a 
peer-reviewed  journal  [6].  In  the  same  peer-reviewed 
journal  it  was,  however,  subsequently  explained  that  the 
philosopher  and  his  team  had  only  correctly  identified  the 
hidden  entities  in  5  of  the  9  tests.  In  the  other  4  cases  they 
had,  without  realising  it,  misclassified  humans  as  machines 
and  machines  as  being  human  [21]. 

In  the  following  sections  we  consider  a  number  of 
transcripts  obtained  from  practical  Turing  tests.  We  refer 
here  to  5-min-long  tests  only  and  show  actual  transcripts 
from  such  tests.  Although  this  is  the  run  time  stated  by 
Turing  himself  [30],  as  indicated  in  the  next  section,  it  is  in 
fact  not  a  critical  issue  with  regard  to  the  main  argument 
raised  in  this  paper.  As  you  will  see,  in  the  tests  carried  out 
there  was  a  hard  cut-off  at  the  end  of  each  discourse  and  no 
partial  sentences  were  transmitted.  Once  a  sentence  had 


been  transmitted  it  could  not  be  altered  or  retracted  in  any 
way.  The  transcripts  appear  exactly  as  they  occurred,  and 
any  spelling  mistakes  and  other  grammatical  errors  are  not 
due  to  poor  editorial  practice. 

In  all  the  two  hidden  entity  (3-participant)  tests  (see 
Fig.  lb)  judges  were  clearly  told  beforehand  that  in  each 
parallel  conversation  one  of  the  hidden  entities  was  human 
and  the  other  was  a  machine.  They  were,  however,  given 
no  indication  as  to  whether  the  LHS  (left-hand  side  of  the 
computer  screen)  or  RHS  would  be  human  or  machine.  On 
the  judges’  score  sheets  each  judge  could  mark  both  the 
LHS  and  RHS  entities  as  being  Human,  Machine  or  they 
could  say  if  they  were  Unsure  [22,  37]. 

Right  Identification 

The  Turing  test  involves  a  machine  which  pretends  to  be  a 
human  in  terms  of  conversational  abilities.  The  “right 
identification”  stated  by  Turing  can  mean  either  that  a 
judge  merely  correctly  identifies  the  machine  or  that  they 
correctly  identify,  at  the  end  of  a  paired  conversation, 
which  was  the  machine  and  which  was  the  human  [27]. 
However,  we  are  not  so  interested  here  with  cases  in  which 
a  judge  mistakes  a  human  for  a  machine.  This  phe¬ 
nomenon,  known  as  the  confederate  effect  [19],  has  been 
discussed  elsewhere  [20,  38,  41].  It  needs  to  be  recognised, 
however,  that  such  a  decision  might  affect  the  judge’s 
decision  regarding  the  machine  being  investigated  in 
parallel. 

The  concept  of  what  is  and  what  is  not  a  “right  identi¬ 
fication”  is  important  as  far  as  a  machine  taking  part  in  the 
Turing  test,  and  the  30  %  pass  mark,  is  concerned,  and  we 
take  a  relatively  strict  approach  in  this  sense.  One  view¬ 
point  is  that  for  a  judge  to  make  the  “right  identification” 
they  must  correctly  identify  both  the  machine  as  being  a 
machine  and  the  hidden  human  as  being  a  human  [27].  This 
means  that  any  other  decision  on  the  part  of  a  judge  would 
not  be  a  “right  identification” ;  this  therefore  includes  cases 
in  which  either  the  machine  is  selected  as  a  human  or  a 
human  is  selected  as  a  machine.  Also  included  are  cases  in 
which  the  judge  is  Unsure  about  either  or  both  entities  as 
the  judge  in  such  cases  has  failed  to  identify  the  machine  as 
a  machine  and/or  the  human  as  a  human — the  right  iden¬ 
tification.  Our  stricter  interpretation  here,  however,  only 
considers  the  cases  in  which  the  machine  was  itself  not 
correctly  identified,  the  judge  stating  either  that  the 
machine  was  a  human  or  that  they  were  Unsure  about  it. 

It  is  also  possible  to  encounter  cases  in  which  a  machine 
was  correctly  identified  as  being  a  machine  but  where  the 
parallel  hidden  human  in  each  case  was  incorrectly  selected 
as  being  a  machine  and/or  the  judge  gave  an  Unsure  mark 
against  the  human,  as  either  of  these  would  not  be  a  right 
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identification.  Such  cases  are  though  troublesome  as  far  as 
the  whole  basis  of  the  test  is  concerned  in  that  a  machine,  if 
so  identified,  would  be  gaining  a  point  not  so  much  because 
of  its  own  competence  but  rather  because  of  the  nature  of 
its  hidden  human  competitor.  However,  it  does  make  things 
slightly  easier  for  the  machine. 


Taking  the  Test 

We  include  in  this  section  several  transcripts  which  arose 
from  the  Turing  tests  which  took  place  at  the  Royal  Society 
in  June  2014.  The  information  given  to  the  judges  on  the 
day,  which  is  true  as  observed  here,  is  that  one  of  the 
hidden  entities  is  a  human  and  the  other  is  a  machine. 
However,  it  was  up  to  the  judge  involved  in  each  case,  and 
now  it  us  up  to  the  reader,  to  decide  which  was  which 
based  on  the  transcripts  shown. 

The  first  of  these  transcript  pairs  we  simply  give  here  for 
you  to  decide  which  was  which.  In  order  for  you  to  check 
on  the  answer  we  provide  the  solution,  along  with  the 
judge’s  decision  on  the  day,  at  the  end  of  the  paper. 


LHS 


RHS 


[11:59:14]  Judge:  Hello 

[11:59:18]  Entity:  I  was  hoping  to  see 
you  Judge! 

[11:59:25]  Judge:  good! 

[11:59:29]  Entity:  That’s  good  to  hear. 

[11:59:38]  Judge:  how  are  you 

[11:59:41]  Entity:  Everything’s  fine, 
Caused  any  scandals  lately? 

[12:00:09]  Judge:  no  not  really! 

[12:00:22]  Entity:  Can  you  use  better 
grammar  and  rephrase  your 
question?  I’d  like  to  help,  but  I  can’t 
even  make  it  through  the  jumbled 
mess.  Use  capitals  and  punctuation. 

[12:00:34]  Judge:  sorry 

[12:00:38]  Entity:  You’re  excused. 

[12:00:49]  Judge:  ‘t  type  as  good  as  I 
talk 

[12:00:55]  Entity:  Talk  to  me 
whenever  you  can. 

[12:01:49]  Judge:  Are  you  interested 
in  artificial  intelligence? 

[12:01:55]  Entity:  I  believe  machines 
should  do  many  to 

assist  the  lives  of  man. 


[11:59:57]  Judge:  How  are 
you  doing? 

[12:00:07]  Entity:  I’m  doing 
fine,  you? 

[12:00:17]  Judge:  great 

[12:00:27]  Entity:  That’s 
good.  :) 

[12:01:07]  Judge:  are  you 
enjoying  this  process? 

[12:01:17]  Entity:  So  far 
seems  good. 

[12:02:02]  Judge:  sorry  if  I 
am  slow 

[12:02:12]  Entity:  That’s  ok. 

[12:02:31]  Judge:  do  you 
have  a  favorite  tv  show? 

[12:02:47]  Entity:  I  have 
quite  a  few. 

[12:03:01]  Judge:  me  too 

[12:03:16]  Entity:  What 
type  ot  TV  do  you  like? 

[12:03:34]  Judge:  Game  of 
thrones 

[12:03:50]  Entity:  I  prefer 
documentories. 


[12:02:12]  Judge:  I  agree 

[12:02:17]  Entity:  You  are  being  well 
behaved  today. 


LHS 


RHS 


[12:02:49]  Judge:  as  well  behaved  as  I 
can  be! 

[12:03:02]  Entity:  Well  then  our 
peanut  distracted  me  from  the 
movie.  Bad  baby.  [I  look  at  your 
stomach  as  I  say  it,  laughing]  I  don’t 
mind  if  -. 

[12:03:19]  Judge:  you’re  funny! 

[12:03:23]  Entity:  Do  you  really  think 
so?  I’ll  always  believe  you’re 
brilliant! 

[12:04:00]  Judge:  thankyou 

[12:04:03]  Entity:  Don’t  think 
anything  of  it! 


In  this  conversation  it  can  be  observed  that  the  judge 
appeared  to  discourse  much  more  with  the  left-hand  entity 
rather  than  that  on  the  right-hand  side.  This  is  something 
that  occurs  fairly  often.  Both  conversations  though  lasted 
for  the  same  5  min  although  clearly  the  LHS  was  more 
comprehensive. 

Now  consider  the  next  parallel  conversation.  In  this  case 
we  give  both  the  solution  and  the  judge’s  decision  at  the 
time  directly  at  the  end  of  the  transcripts. 


LHS 


RHS 


[16:28:55]  Judge:  how  tall  are 
you 

[16:29:18]  Entity:  almost  6 
feet  tall 

[16:29:46]  Judge:  how  heavy 
are  you 

[16:30:12]  Entity:  just  over 
ten  stone 

[16:30:47]  Judge:  what  is  your 
gender 

[16:30:59]  Entity:  male 

[16:31:45]  Judge:  what  is  your 
bmi 

[16:31:54]  Entity:  i  have  no 
clue 

[16:32:41]  Judge:  what  are 
you  aware  of  at  this  moment 

[16:32:57]  Entity:  a  lot  of 
things 

[16:33:17]  Judge:  how  old  are 
you 

[16:33:36]  Entity:  26 


[16:29:04]  Judge:  how  tall  are  you 

[16:29:10]  Entity:  My  height  is 
about  160  cm — it’s  5  feet  4  inches. 

[16:29:37]  Judge:  how  heavy  are 
you 

[16:29:59]  Entity:  My  grandfather 
taught  me  not  to  disclose  non- 
solicited  information  about  myself. 
Like  on  the  question  “Do  you  have 
watch” — never  tell  time,  but 
answer  “Yes”  or  “No”.  I  always 
follow  his  advice.  And  I  forgot  to 
ask  you  where  you  are  from... 

[16:31:09]  Judge:  what  is  your 
gender 

[16:31:17]  Entity:  I’m  a  young  boy, 
if  you  care  to  know.  And  not  very 
ugly,  by  the  way! 

[16:32:26]  Judge:  what  are  you 
aware  of  at  this  moment 

[16:32:37]  Entity:  I  would  rather  not 
talk  about  it  if  you  don’t  mind. 
Could  you  tell  me  what  are  you?  I 
mean  your  profession. 

[16:33:09]  Judge:  how  old  are  you 


Springer 


414 


Cogn  Comput  (2016)  8:409-419 


LHS 


RHS 


LHS 


RHS 


[16:33:12]  Entity:  I’m  a  little  boy 
13  years  old. 

[16:33:38]  Judge:  what  are  your 
qualifications 

[16:33:43]  Entity:  My 
qualifications?  I’m  not  gonna  tell 
you.  Oooh.  Anything  else? 


In  this  conversation  the  LHS  entity  was  a  senior  male 
human,  whereas  the  RHS  was  the  machine  Eugene 
Goostman.  This  was  in  fact  one  of  the  ten  transcripts 
responsible  for  Eugene  passing  the  Turing  test  in  June 
2014.  The  judge  decided  that  the  LHS  was  indeed  human, 
although  they  were  unable  to  give  any  further  details. 
However,  they  also  decided  that  the  RHS  (Eugene)  was  a 
human. 

The  conversation  with  the  human  entity  was,  it  can  be 
observed,  relatively  boring,  merely  being  a  case  of  question 
and  answer  with  limited  responses.  Eugene  did  very  well 
here  though  as  the  judge  was  an  expert  on  machines/ 
robotics  and  was  well  aware  of  machine  conversations.  The 
fact  that  Eugene  convinced  such  a  person  is  quite  an 
accomplishment.  Eugene  tried  to  power  the  conversation 
by  asking  the  judge  questions.  At  first  the  judge  was  not 
having  any  of  it  and  simply  ignored  Eugene’s  question, 
even  though  this  was  rather  rude.  Eugene  persevered, 
however,  and  eventually  the  judge  gave  in  and  responded. 
Generally  there  was  more  content  in  Eugene’s  conversation 
than  that  with  the  human  hidden  entity. 

One  aspect  of  the  test  is  its  reliance  on  the  interrogators 
to  do  a  good  job.  It  is  the  interrogator  who  conducts  and 
drives  each  conversation  and  who  also  makes  the  decision 
on  which  entity  was  the  machine  and  which  was  the 
human.  Some  researchers  have  suggested  this  as  a  weak 
point  of  the  test  [9]  although  we  believe  that  it  is  an 
important  part  of  the  test  as  Turing  set  it  out.  However, 
here  the  quite  direct  question  and  answer  attempt  appears 
to  have  failed  to  out  the  machine. 

The  transcript  that  follows  takes  on  a  slightly  different 
style: 


answer!  Could  you  tell  me  what 
are  you?  I  mean  your  profession. 

[16:31:55]  Judge:  I  was  more 
interested  in  you  guessing 
whether  I  am  male  or  female 

[16:31:59]  Entity:  I  like  talking  to 
girls!  Oooh.  Anything  else? 

[16:32:54]  Judge:  but  that  means 
nothing — males  and  females 
like  talking  to  ‘girls’,  but  only 
males  call  women  girls? 

[16:33:06]  Entity:  yeah,  nothing. 
You  are  right!  Where  do  you 
came  from,  by  the  way?  Could 
you  tell  me  about  the  place 
where  you  live? 

[16:34:00]  Judge:  I  live  in  a  prison 
of  abstractions 


[16:33:32]  Judge:  so  you  need  to 
guess  if  *1*  am  male  or  female 

[16:34:21]  Entity:  you  have  to 
be  male  or  female 

[16:34:34]  Judge:  or  computer 


[16:34:06]  Entity:  Tell  me  more 
about  Prison.  I  like  to  learn 
about  different  places! 


In  this  case  the  LHS  was  in  fact  the  machine  Eugene, 
whereas  the  RHS  was  a  male  human.  The  judge  in  this  case 
decided  that  the  LHS  was  a  male  human,  non-native 
English  speaking  and  a  teenager.  They  were  also  definite 
that  the  RHS  was  a  machine. 

It  is  one  of  those  transcripts  where  quite  simply  there 
was  a  lot  more  going  on  in  the  Eugene  conversation;  it  had 
the  tendency  to  draw  the  interrogator  to  it,  whereas  there’s 
not  much  at  all  in  the  human  case.  In  fact  the  hidden  human 
may  well  have  not  done  well  for  themselves  by  claiming  no 
knowledge  about  the  Turing  test  early  on.  Possibly  incor¬ 
rect  decisions  can  be  made  by  interrogators  based  on  an 
assumption  that  everyone  must  know  a  particular  piece  of 
information  [34].  In  this  case  though,  as  the  event  was  a 
Turing  test  session  they  appear  to  have  some  quite  strong 
evidence.  It  probably  goes  to  show  that  you  cannot  rely  on 
the  knowledge  base  of  humans. 

In  the  next  transcript  we  again  give  both  the  solution  and 
the  judge’s  decision  at  the  time  directly  at  the  end  of  the 
transcripts. 


LHS 


RHS 


LHS 


[16:30:37]  Judge:  don’t  you  think 
the  imitation  game  was  best  in  it 
it’s  original  form? 

[16:30:49]  Entity:  Oh,  your 
questions  seems  to  be 
rhetorical — you  do  already 
know  the 


RHS 


[16:31:08]  Judge:  don’t  you 
thing  the  imitation  game  was 
more  interesting  before  Turing 
got 

to  it? 

[16:32:03]  Entity:  I  don’t  know. 
That  was  a  long  time  ago. 


[14:49:59]  Judge:  Hi  what’s  your 
name? 

[14:50:27]  Entity:  i  dont  kno 
[14:50:49]  Judge:  My  name’s 
Jane.  I  live  in  London.  Wehre 
do  you  live 


[14:50:07]  Judge:  Hello  where 
are  you? 

[14:50:10]  Entity:  Hi  there  Judge. 
[14:50:20]  Judge:  Where  are 
you? 
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LHS  RHS 


[14:50:27]  Entity:  I’m  in  a  room 
not  too  far  away  from  where 
you  are. 

[14:50:58]  Judge:  What  colour  is 
the  carpet  in  your  room 

[14:51:03]  Entity:  I  like  pink; 
what  do  you  like? 

[14:51:24]  Judge:  I  like  pink.  My 
name  is  Jane.  What’s  your 
name? 

[14:51:31]  Entity:  Red.  Hello 
Jane.  Its  nice  to  meet  such  a 
fine  lady  as  yourself. 

[14:52:00]  Judge:  Red  is  my 
favourite  colour.  What’s  yours? 

[14:52:09]  Entity:  My  favorite 
color  is  red. 

[14:52:26]  Judge:  You  use 
AMerican  spelling.  Are  you  in 
the  USA? 

[14:52:39]  Entity:  But  you  are 
not  even  American.  Just 
remember  atheism  is  only  a 
belief  system.  And  it  too  can  be 
forced  on  people  who  don’t 
agree. 

[14:53:01]  Judge:  Who  told  you  I 
was  an  atheist? 

[14:53:06]  Entity:  Everything 
about  you  screams  warning: 
Child’. 

[14:53:46]  Judge:  But  I  come  in 
peace  to  your  planet.  How  do 
you  like  your  eggs — over  easy 
or  sunny  side  up? 

[14:53:56]  Entity:  Sometimes  I 
can’t  help  but  show  you  my 
professional  planet.  How  I 
wonder  if  I  like  my  eggs-over 
easy  or  sunny  side  up?  This  is 
so  explainable.  That  list  makes 
me  feel  unsure  Jane. 

[14:54:40]  Judge:  Well  eggs  are 
very  good  indicators  of 
personal  taste.  Do  you  have  a 
favourite  food? 

[14:54:47]  Entity:  Juicy  red 
seedless  grapes  taste  so  very 
good. 


Here  on  the  LHS  it  was  a  hidden  human  entity,  whereas 
on  the  RHS  it  was  the  machine  JFred.  The  judge  con¬ 
cluded,  however,  that  on  the  LHS  it  was  a  machine  and  felt 
that  the  entity  exhibited  very  poor  human-like  conversa¬ 
tion.  On  the  other  hand  the  judge  was  confident  that  the 


RHS  (the  machine  JFred)  was  a  male  human  who  was  most 
likely  an  American. 

The  judge’s  decision  in  terms  of  the  LHS  entity  was  not 
particularly  surprising.  The  hidden  human  entity  was  asked 
on  more  than  one  occasion  what  their  name  was  to  which 
they  replied  “I  don’t  know”.  As  a  result  the  judge  spent 
much  more  time  conversing  with  the  machine  on  the  RHS. 
This  is  a  particular  aspect  of  the  test  that  it  involves  a  direct 
comparison  between  a  machine  and  a  human,  rather  than 
merely  a  machine  conversing  on  its  own.  Here  we  can  see 
that  the  hidden  human  involved  was  quite  simply  relatively 
poor  at  conversation  and  this  helped  the  cause  of  the 
machine. 

Alternative  Views 

There  are  many  different  interpretations  of  Turing’s  imi¬ 
tation  game,  and  much  controversy  has  arisen  as  to  which 
of  these,  if  any,  was  Turing’s  own  intended  version  [15]. 
The  vast  majority  appear  to  view  the  game  in  the  form  of 
what  is  commonly  known  as  the  “Standard  Turing  Test” 
[26],  and  this  is  the  interpretation  taken  here.  It  is  a  literal 
interpretation  based  essentially  on  what  Turing  actually 
said  in  his  presentations  and  his  1950  paper  and  without 
recourse  to  tangential  connections  and/or  pure  conjecture 
on  what  a  paper’s  author  believes  that  Turing  really  meant 
to  say. 

We  acknowledge  as  examples  of  this,  that  some  see  it  as 
being  something  to  do  with  artistic  and  emotional  intelli¬ 
gence  [24],  whereas  others  deem  it  to  be  concerned  with 
modelling  the  human  mind  by  generating  its  verbal  per¬ 
formance  capacity  [8].  Others  meanwhile  regard  it  in  terms 
of  considering  the  gender  aspect,  the  sex  of  the  human  foil 
being  important  in  the  test  [7,  9,  12,  26].  None  of  these 
views,  however,  do  we  see  as  indicating  the  test  to  be 
detrimental  to  the  human  race. 

However,  we  then  have  the  Shanahan  view,  quoted  by 
his  own  University  news  as:  “Turing  also  didn’t  say  a 
5-min  test  would  mean  success  achieving  human-level  AI; 
for  that,  he  would  require  much  longer  conversations”  [10]. 
The  point  being  here  not  whether  the  test  is  a  5-min  one  or 
a  20-min  one  but  rather  that  in  the  mind  of  Shanahan  there 
is  some  time  for  which  a  machine  could  successfully 
converse  that  would  indicate  that  its  intelligence  has 
reached  human-level. 

Unfortunately  Shanahan  is  not  a  lone  voice.  Consider  if 
you  will:  “Hunch  CEO  Chris  Dixon  tweeted,  ‘The  point  of 
the  Turing  Test  is  that  you  pass  it  when  you’ve  built 
machines  that  can  fully  simulate  human  thinking.’  No,  that 
is  precisely  not  how  you  pass  the  Turing  test.  You  pass  the 
Turing  test  by  convincing  judges  that  a  computer  program 
is  human”  [2].  Interestingly  it  is  the  emulation  of  human 


[14:51:08]  Entity:  i  live  in  london 
aswell 

[14:51:41]  Judge:  Do  you  like 
living  in  London? 

[14:52:32]  Entity:  yes  i  like  it 
because  there  are  a  lot  of  things 
to  do  there  and  also  it  is  the 
capital  city  of  England 

[14:52:44]  Judge:  What’s  my 
name? 

[14:52:53]  Entity:  sorry  ? 

[14:53:10]  Judge:  What’s  my 
name? 

[14:53:14]  Entity:  jane 

[14:53:54]  Judge:  And  what’s 
your  name? 

[14:54:08]  Entity:  i  dont  know 

[14:54:53]  Judge:  Would  you  like 
me  to  call  you  Alan? 
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intelligence,  in  a  machine,  that  Kurzweil  picks  up  on  as 
being  the  tipping  point  [11]. 

Then  there  are  those  who  (somehow)  read  all  sorts  of 
concepts  into  the  Turing  test,  telling  us  what  Turing  actu¬ 
ally  had  in  mind  with  his  test  even  if  he  didn’t  tell  us 
himself:  “Alan  Turing  himself  envisioned — a  flexible, 
general-purpose  intelligence  of  the  sort  that  human  beings 
have,  which  allows  any  ordinary  individual  to  master  a  vast 
range  of  tasks,  from  tying  his  shoes  to  holding  conversa¬ 
tions  and  mastering  tenth-grade  biology”  [13]. 

From  these  voices  it  is  clear  that  there  is  a  school  of 
opinion  that  associates  a  Turing  test  pass  with  human-level 
intelligence.  We  accept,  in  Shanahan’s  case,  that  there  is  a 
question  about  the  actual  duration  of  the  conversation 
involved.  However,  we  would  argue  that  to  be  of  little 
importance  in  comparison  with  the  big  picture  issues  that 
are  at  stake  here. 

Silence 

In  this  section  we  explain  briefly  how  it  is  quite  possible  for 
a  machine  to  pass  the  Turing  test  not  by  its  apparent  skill  at 
human  conversation  but  rather  by  simply  remaining  silent 
throughout  [40].  Rather  than  being  a  mere  theoretical  or 
philosophical  quirk  it  turns  out  that  in  fact  passing  the 
Turing  test  in  this  way  also  has  an  underlying  practical 
basis  to  support  it  with  numerous  examples  to  boot. 

Turing  said  that  in  the  test  a  machine  had  to  try  and 
pretend  to  be  a  man  (although  now/here  we  take  that  to 
mean  human).  In  his  1950  paper  he  also  pointed  to  the  fact 
that  at  the  end  of  5  min  the  judge  had  to  make  a  decision  as 
to  the  nature  of  the  entity.  If  they  made  the  right  identifi¬ 
cation  and  correctly  identified  the  machine  then  this  would 
effectively  be  a  point  against  the  machine,  whereas  if  the 
judge  either  thought  that  the  machine  was  a  human  or  if 
they  were  Unsure  as  to  its  nature  then  this  would  be  a 
wrong  identification  and  would  be  a  point  for  the  machine. 
The  pass  mark  for  a  machine  in  the  test  was  set  by  Turing 
to  be  3  or  more  points  out  of  every  10  [30]. 

But  here  we  face  a  critical  issue,  what  if  a  machine  was 
to  remain  silent?  The  basic  nature  of  the  test  is  that  a 
machine,  by  conversing,  fails  the  test  by  giving  themselves 
away  as  clearly  being  a  machine.  So  if  they  remain  silent 
they  cannot  give  themselves  away. 

If  a  machine  remains  completely  silent  during  a  5 -min 
conversation  a  judge  receives  no  response  to  any  of  their 
questions  or  discussion  from  the  hidden  entity  and  there¬ 
fore,  in  theory  at  least,  cannot  not  make  the  right  identifi¬ 
cation  and  definitely  say  that  they  have  been  conversing 
with  a  machine.  It  would  not  be  expected  that  a  judge, 
under  such  circumstances,  would  categorise  the  silent 
entity  as  being  a  human,  although  that  is  a  possibility,  the 


most  likely  case  is  for  the  judge,  as  we  have  seen  in  the 
practical  examples,  to  give  an  “Unsure”  response.  This  of 
course  is  not  a  right  identification  and  is  therefore  a  point 
for  the  machine. 

It  is  thus  quite  possible  for  a  machine  to  simply  remain 
silent  to  any  utterances  of  a  judge  and  to  pass  the  Turing 
test  if  at  least  3  out  of  10  judges  as  a  result  either  rate  the 
machine  as  being  a  human  or  indicate  that  they  are  unsure. 
The  only  thing  acting  against  such  a  strategy  is  the  fact  that 
the  machine  is,  in  each  conversation,  competing  against  a 
human  and  if  the  judge  is  certain  that  the  other  (hidden) 
entity  is  a  human  then  they  can  deduce  that  therefore  the 
silent  entity  must  be  a  machine.  Conversely  in  practice 
many  humans  are  actually  categorised  as  machines  in  such 
tests  [38].  Therefore,  it  is  also  potentially  possible  that  a 
(silent)  machine  can  be  categorised  as  being  human  mainly 
because  their  hidden  human  competitor  is  categorised  by 
the  judge  as  being  a  machine. 

We  now  give  an  example  of  a  transcript  in  which  a 
machine  simply  did  not  respond.  This  particular  “conver¬ 
sation”  occurred  during  the  Turing  tests  held  at  the  Royal 
Society  in  June  2014  between  a  judge  and  the  machine 
Cleverbot.  At  the  end  of  the  conversation  the  judge  was  not 
able  to  identify  the  hidden  entity  as  being  a  machine,  i.e.  they 
did  not  make  the  right  identification,  deciding  that  they  were 
“unsure” .  It  is  straightforward  to  see  that  there  quite  simply 
was  not  enough  information  for  the  judge  to  go  on. 

Example  transcript 

[10:58:08]  Judge:  good  day 

[10:58:08]  Entity: 

[10:58:46]  Judge:  is  no  response  an  answer 

[10:58:46]  Entity: 

[10:59:35]  Judge:  am  i  not  speaking  you’re  language 

[10:59:35]  Entity: 

[11:00:25]  Judge:  silence  is  golden 

[11:00:25]  Entity: 

[11:01:32]  Judge:  shhh 

[11:01:32]  Entity: 

[11:03:07]  Judge:  you  make  great  conversation 

[11:03:07]  Entity: 

As  far  as  we  are  aware,  the  silence  on  the  part  of  the 
machine  in  this  transcript  was  caused  by  a  technical  fault 
rather  than  any  decision  (conscious  or  otherwise)  on  the 
part  of  the  machine.  That  said,  it  is  perhaps  a  quirk  with  the 
Turing  test,  as  described  by  Turing,  that  it  is,  in  theory  at 
least,  quite  possible  for  a  machine  to  pass  the  test  by 
remaining  silent  throughout.  Essentially  the  machine 
makes  no  utterances  which  give  the  game  away  that  they 
are  a  machine  and  hence  the  judges  involved  have  no 
evidence  to  use  against  them.  This  whole  issue  of  the 
strategy  of  silence  is  discussed  at  length  in  Warwick  and 
Shah  [40]. 
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The  example  given  here  is  just  that,  an  example,  as 
there  are  numerous  other  cases  reported  on  in  Warwick 
and  Shah  [40].  An  interesting  feature  is  the  response  of 
the  interrogators  involved  with  those  particular  transcripts. 
In  each  case  the  interrogator  has  been  a  different  person 
yet  their  responses  have  been  remarkably  similar.  Essen¬ 
tially  they  have  all  judged  the  hidden  entity  on  the  evi¬ 
dence  of  the  transcript  in  front  of  them  and  have  not  been 
swayed  by  the  other  parallel  conversation  they  were 
involved  with,  although  that  might  have  taken  more  of 
their  attention  due  to  the  machine’s  silence.  So,  in  prac¬ 
tice,  interrogators  appear  to  state  that  they  are  unsure 
about  the  silent  entity,  thereby  supporting  the  argument 
given  in  this  section. 

As  far  as  the  Turing  test  is  concerned,  however,  if  a 
machine  remains  silent  and  passes  the  test  then  of  course 
this  could  have  been  due  to  the  fact  that  the  machine  was, 
for  example,  switched  off  or  perhaps  wasn’t  even  there  at 
all.  For  someone  to  make  the  link  between  a  switched-off 
computer  and  human-like  intelligence  is  frankly  ridiculous 
in  the  extreme.  In  fact  to  link  any  level  of  intelligence  with 
a  switched-off  computer  is  not  sustainable.  Otherwise, 
switch  the  computer  on  and  its  level  of  intelligence  drops — 
clearly  this  is  contrary  to  what  we  witness. 

So  we  have  here  the  Shanahan/Kurzweil  argument  that 
the  fact  that  a  computer  was  unplugged  when  subjected  to  a 
series  of  Turing  tests,  whether  they  are  of  5-min  duration 
or,  simply  to  please  Shanahan,  lasting  for  30  min,  means 
that  the  human  race  will  come  to  an  end.  Whoever  is 
responsible  for  unplugging  the  machine  clearly  has  a  lot  to 
answer  for. 

Discussion 

Similarly  to  the  opening  of  his  1948  paper  “I  propose  to 
investigate  the  question  as  to  whether  it  is  possible  for 
machinery  to  show  intelligent  behaviour”  [29]  in  which 
Turing  introduced  an  imitation  game,  Turing,  perhaps 
mischievously  (we  will  never  know),  started  his  1950  paper 
by  considering  whether  machines  could  think.  Replacing 
this  question  with  a  conversational  imitation  test,  the 
concept  being  that  if  a  machine  could  do  sufficiently  well 
(or  rather  not  do  so  badly)  at  his  test,  dare  we  say  here  to 
pass  the  Turing  test,  then  we  would  have  to  concede  that  it 
was  a  thinking  machine.  In  a  direct  way,  whatever  the  pass 
mark  and  whatever  the  exact  rules  and  nature  of  his  test,  it 
became  a  direct  practical  replacement  for  a  much  more 
philosophical  question  regarding  the  thinking  process.  On 
the  other  hand  for  a  machine  to  fail  the  test  we  would  have 
to  concede  that  it  is  not  a  thinking  entity.  So  can  we  say 
that  if  a  machine  passes  the  Turing  test  it  is  a  thinking 
entity? 


Well  whatever  thinking  is,  it  is  certainly  a  property  of 
each  and  every  human  brain  that  exists  within  a  human  body. 
We  wish  to  exclude  from  the  argument  here  brains,  con¬ 
sisting  of  human  neurons,  which  are  grown  and  placed 
within  a  robot  body  [33]  for  no  better  reason  than  they 
complicate  the  argument.  The  assumption  from  the  inexpe¬ 
rienced  Turing  tester  might  be  that  a  human,  acting  as  a 
hidden  entity,  machine  foil,  would  be  expected  to  pass  the 
Turing  test  on  a  regular  basis  as  long  as  they  are  simply 
themselves.  It  might  be  thought  that  occasionally  they  might 
be  classified  as  a  machine  by  a  poor  judge  but  that  this  would 
be  an  odd  occurrence  and  almost  surely  the  vast  majority  of 
judges  would  classify  them  as  being  human.  Unfortunately 
this  is  far  from  the  truth.  Indeed  numerous  humans  have  been 
classified  at  different  times  as  being  a  machine  [38]. 

In  the  example  transcripts  it  was  shown  how  a  machine 
can  be  thought  to  be  human  because  of  its  communication 
abilities,  but  also  when  a  hidden  human  does  not  com¬ 
municate  so  well  this  can  in  fact  assist  a  machine  in  its 
goal.  In  the  second  set  of  transcripts  we  could  see  the 
machine  Eugene  Goostman  at  work.  Eugene  achieved  the 
30  %  pass  mark  in  the  tests,  the  full  set  of  transcripts  to 
achieve  that  appearing  in  Warwick  and  Shah  [39].  In  this 
particular  transcript  case  both  the  hidden  human  and 
Eugene  were  classified  as  being  human.  This  is  an  inter¬ 
esting  point  because  even  when  judges  are  specifically  told 
that  one  entity  is  a  machine  and  the  other  is  a  human  it  is 
frequently  the  case  that  their  final  decision  is  other  than  a 
simple  human/machine  pairing. 

Conclusions 

It  is  fairly  clear  to  see  that  when  the  test  was  set  up  in  1950 
such  skills  as  a  machine  fooling  people  into  believing  that 
it  is  a  human  through  a  short  communication  exercise 
would  have  been  very  difficult  for  most  people  to  under¬ 
stand.  However,  in  introducing  the  test,  Turing  linked  it 
inextricably  with  the  concept  of  thinking  and  there  is  a  nice 
philosophical  argument  in  consequence  concerning  how 
one  can  tell  if  another  human  is  thinking.  This  was  a 
brilliant  link  by  Turing  which,  as  a  result,  has  brought 
about  a  multitude  of  arguments  between  philosophers  and 
AI  researchers  as  to  the  test’s  meaning  and  gravity. 

But  Turing’s  game  has  extended  way  beyond  the  ivory 
towers  of  academe  and  has  a  truly  popular  following.  As  an 
example  the  Wikipedia  “Turing  Test”  page  typically 
receives  2000-3000  views  every  day  at  present.  On  1  day, 
9  June  2014,  after  it  was  announced  that  the  Turing  test  had 
been  passed,  the  same  page  received  a  total  of  71,578 
views,  an  amazing  figure.  As  a  comparison,  top  Wikipedia 
pages  such  as  “Leonardo  DiCaprio”  and  “The  Beatles” 
received  respectively  only  11,197  and  10,328  views  on  that 
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same  day.  But  with  this  popular  following  has  come  mis¬ 
conceptions  as  to  what  the  test  is  about  and,  in  particular  a 
sort  of  folklore  mythology  has  arisen  that  the  Turing  test  is 
a  test  for  human-like  intelligence.  As  we  have  seen,  this 
folklore  has  been  fuelled  by  some  academics  and  technical 
writers  who  perhaps  have  not  read  the  works  of  Turing  as 
thoroughly  as  they  should. 

Let  us  be  clear,  the  Turing  test  is  not,  never  was  and 
never  will  be  a  test  for  human-level  or  even  human-like 
intelligence.  Turing  never  said  anything  of  the  sort  either  in 
his  papers  or  in  his  presentations.  The  Turing  test  is  not, 
never  was  and  never  will  be  a  test  for  human-level  think¬ 
ing.  Turing  didn’t  say  that  either. 

The  Turing  test  does  require  a  machine  taking  part  to 
condemn  itself  by  what  it  says,  as  judged  subjectively  by  the 
human  interrogator.  Alternatively  if  a  machine  does  not  give 
itself  away  on  a  sufficient  number  of  occasions  it  could  result 
in  a  machine  “passing  the  Turing  test”,  in  the  extreme  case 
simply  by  remaining  silent.  Of  course,  this  does  beg  the 
question,  what  exactly  does  it  mean  to  pass  the  Turing  test? 

Earlier  in  the  paper  we  considered  that  Turing  introduced 
his  imitation  game  as  a  replacement  for  the  question  “Can 
machines  think?”  [30].  The  end  conclusion  by  many  as  a 
result  of  this  is  that  if  a  machine  passes  the  test  then  we  have 
to  regard  it  as  a  thinking  machine.  Turing  clearly  dissociated 
the  way  a  machine  thinks  from  the  human  version.  He  said 
“May  not  machines  carry  out  something  which  ought  to  be 
described  as  thinking  but  which  is  very  different  from  what  a 
man  does?”  [30].  So  even  human-like  thinking  for  machines 
was  not  on  the  radar  as  far  as  Turing  was  concerned.  He  also 
said  in  reference  to  the  year  2000,  “one  will  be  able  to  speak 
of  machines  thinking  without  expecting  to  be  contradicted” 
[30].  Noam  Chomsky  wondered  that  of  all  the  ways  a 
machine  could  display  intelligence  why  did  Turing  choose  a 
test  involving  human  language  [3]  which  is  merely  one  small 
part  of  human  intelligence. 

The  Turing  test  is  a  simple  test  of  a  machine’s  commu¬ 
nication  ability.  It  is  interrogated  by  a  human  and  is  directly 
compared  with  another  human  in  a  parallel  fashion  with 
regard  to  human  communication  abilities.  In  that  sense  it 
merely  involves  one  aspect  of  human  intelligence,  as  pointed 
out  by  Chomsky.  If  a  machine  passes  the  Turing  test  it 
exhibits  a  capability  in  communication.  This  does  not  in  any 
terms  mean  that  the  machine  displays  human-level  intelli¬ 
gence  or  consciousness.  So  even  if  Kurzweil  is  correct  in  his 
prediction,  for  a  machine  to  pass  the  Turing  test  does  not 
mean  that  the  end  of  humanity  is  just  around  the  comer. 

Solution 

Here  we  provide  a  solution  to  the  first  of  the  Transcripts 
included  in  the  “Taking  the  Test”  section  which  took  place 
between  a  human  interrogator  and  two  hidden  entities.  The 


LHS  entity  was  in  fact  the  machine/program  Ultra  Hal, 
whereas  the  RHS  entity  was  an  English-speaking  male. 
Meanwhile  whilst  the  judge  correctly  identified  that  the 
LHS  entity  was  a  machine  they  were  unsure  about  the  RHS 
entity  based  on  the  transcripts  shown. 
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